add B300 vLLM AgentX single-node MiniMax-M3 FP4 EAGLE3-GQA MTP / 新增 B300 vLLM AgentX 单节点 MiniMax-M3 FP4 EAGLE3-GQA MTP - #2328

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs

感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have。

如需进行 PR 验证，请为此 PR 添加 full-sweep-fail-fast 标签（强烈推荐）— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30120171478
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30120171478

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30121331788
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30121331788

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30309054681
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30309054681

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30326099097
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30326099097

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30375457395
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30375457395

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30394805725
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30394805725

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30839011398
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30839011398

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30849233121
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30849233121

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30849838984
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30849838984

/reuse-sweep-run 30849838984

/stage-results 30849838984

@cquil11 staged run 30849838984: https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-08-03~r30849838984

This run remains available across future /stage-results requests. Staging the same run ID again updates its staged data. Staging workflow

@cquil11 已将运行 30849838984 发布到预发布环境：https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-08-03~r30849838984

后续的 /stage-results 请求不会移除此运行；再次发布相同的运行 ID 会更新其预发布数据。预发布工作流

As a PR reviewer and CODEOWNER, I have reviewed this and have:

• Verified that as of the moment of typing this, this is the latest version of PR_REVIEW_CHECKLIST.md
• Verified that the general code quality meets the InferenceX standard and does not make the code quality any worse.
• Verified that this PR has passed PR validation. Please link to GitHub Action workflow that shows this.
• Verified that this PR passes evals. Please link to GitHub Action workflow that shows this.
• Verified that speculative decoding PRs uses chat templates to align the AL distribution to real world
• For agentic workloads: verified that speculative-decoding configs (EAGLE / MTP / draft models) run with simulated synthetic acceptance, with the acceptance-length value taken from the committed golden AL curve in golden_al_distribution/ for that model, thinking mode, and draft length. A submission may choose any supported draft length, but it may not substitute a different acceptance target.
• Verified against the current MODELS.md that this PR does not submit a deprecated model, scenario, or model-scenario combination.
• Verified that the model architecture isn't changed with benchmark hacks like using --hf-overrides to skipping indexer for every x layers on models that don't natively support this. As a general rule, we won't accept optimizations that reduces the number of model architecture FLOPs. Anything that makes that same computation run faster is fair game; FLOPs at lower precisions is fine, given that the config passes private evals. As an general north star princple, we should only use optimizations which is used in production by customers that care about accuracy
• If an company claims that they support vLLM/SGLang as first class LLM inference engines on their hardware, I have verified that the respective vLLM submission made using upstream https://hub.docker.com/u/vllm docker repo, upstream SGLang https://hub.docker.com/u/lmsysorg docker repo. The only exceptions are for new hardware, such as MI455X UALoE72, Vera Rubin NVL72, Rubin NVL8, etc., and for new model architectures where there is an actual reason why vLLM/SGLang does not fundamentally support them yet as supported by vLLM/SGLang community maintainers
• If an company claims that they support vLLM/SGLang as first class upstream in-tree LLM inference engines on their hardware, I have have verified that the respective vLLM/SGLang submission has been made before additional frameworks (TRT-LLM, ATOM, etc.). The only exceptions are for new hardware, such as MI455X UALoE72, Vera Rubin NVL72, Rubin NVL8, etc., and for new model architectures where there is an actual reason why vLLM/SGLang does not fundamentally support them yet.
• Verified that every single-node vLLM/SGLang recipe in this PR is documented in the official vLLM recipes and/or the SGLang cookbook:
  • I linked the corresponding upstream PR in the vLLM recipe repo or SGLang repo and verified that it is MERGED before this InferenceX PR merges. An opened, draft, or closed-without-merge upstream PR does not satisfy this requirement. If the matching recipe was already published, I linked the published recipe/cookbook page in the additional detail section below.
• Verified that this PR does not patch the inference engine or serving stack — the pinned image must run as shipped. This covers .patch files / git apply / patch, inline patches embedded in benchmark scripts (e.g. a python3/sed heredoc that rewrites installed engine sources before serving), in-place edits of site-packages, monkey-patching, overwriting container files, and installing forked/rebuilt engine wheels on top of the pinned image. The only exception is a patch covered by a filled-out waiver at docs/waiver/<PR_NUMBER>.md — named after the PR that introduces the patch and filed in that same PR, stating what is patched, why the unmodified upstream image cannot run this benchmark, the upstream PR/issue link, and the removal plan — which I have linked below in the additional detail section.
• If any of the above criteria cannot reasonably be satisfied, I have provided additional reasoning below.

Additional detail section:

• Head reviewed: 3c6efea449ec6e386451bdeddf1880a36eec4c48.
• Items 3 and 4 (PR validation / evals) — UNCHECKED, and this is the blocking gap. The only Run Sweep at this head, run 30877891720, is a reuse no-op: reuse-sweep-gate succeeded and every agentic /, agentic eval /, eval /, and collect-* job is skipped, so it establishes nothing on its own. The real evidence is the pinned source run 30849838984 on head 8767d90c, authorized by /reuse-sweep-run 30849838984 from cquil11. That run is genuinely green — all 16 agentic / lanes plus eval-only and collect-evals executed and succeeded, and its 16 lanes match this config's current search space exactly (TP4 ×6, TP2 ×3, TP4+dram ×7). This PR's own configs/nvidia-master.yaml block and benchmarks/single_node/agentic/minimaxm3_fp4_b300_mtp.sh are byte-identical between 8767d90c and this head. However, utils/aiperf is not: the sweep ran on 818c3a5a, while this head pins b7b16cf8 (agentx-v1.0.1) via main commit 65e0b1ce (fix(agentx): bump AIPerf to v1.0.1 #2484), which by its own description "adds the 98% profiling metric coverage check so incomplete AgentX runs fail instead of passing." The reused numbers were therefore produced by a harness that could not fail an incomplete AgentX run, and the current tree pins one that can. I am not attesting that this submission passes validation under the harness it now ships with. Re-running the sweep at this head under AIPerf v1.0.1 would settle both items.
• Items 11 / 11a (single-node recipe documentation) — UNCHECKED. This is a single-node submission (multinode: false, framework vllm), so the recipe-documentation requirement applies in full. The PR body describes a "day-zero single-node agentic-coding recipe" but links no upstream vLLM recipes PR and no published recipe page, and I found none. The checklist offers a day-zero exception for the upstream image and for framework ordering, but not for recipe documentation; it requires a MERGED upstream PR before this PR merges.
• Item 7 (MODELS.md) — verified, newly added to the checklist. MODELS.md lists MiniMax-M3 (minimaxm3) with active scenario "Agentic coding (EAGLE3 only from 2026-08-03)", and its deprecation table deprecates the non-EAGLE3 agentic arm while publishing the EAGLE3 arm. This PR is the EAGLE3 arm (Inferact/MiniMax-M3-EAGLE3-GQA, method: eagle3), i.e. the surviving published arm — not a deprecated model, scenario, or combination.
• Item 6 (agentic golden AL) — verified. SYNTHETIC_ACCEPT_LEN=2.78 with NUM_SPEC_TOKENS=3 matches golden_al_distribution/minimaxm3_eagle3_gqa.yaml → minimax-m3.thinking_on[3] = 2.78 on current main, and the server runs --default-chat-template-kwargs '{"thinking_mode":"enabled"}', matching that curve's recorded thinking mode. The config uses rejection_sample_method: "synthetic", i.e. simulated synthetic acceptance as required. The script's comment correctly notes the non-GQA curve reads 2.83 at the same level and is not what this runs.
• Item 5 (spec-decode chat templates) — verified: the server sets --default-chat-template-kwargs '{"thinking_mode":"enabled"}' and the workload is agentic trace replay, which is natively chat-formatted.
• Item 8 (architecture): no --hf-overrides, no layer/indexer skipping, nothing that reduces model-architecture FLOPs; the flags are attention backend, KV offload, cudagraph, and scheduling knobs.
• Items 9 / 10 (upstream image and framework ordering): the image is vllm/vllm-openai:nightly-5e35a6f4f9bbc217c599692157ca985c894373f7 from the upstream vLLM Docker org, and this is the vLLM submission for this model/hardware.
• Item 12 (no engine patches): the benchmark script contains no .patch files, git apply/patch calls, sed or heredoc rewrites of engine sources, site-packages edits, or forked engine wheel installs. No waiver is required.
• Open reviewer thread (non-checklist): functionstackx has an outstanding CHANGES_REQUESTED review (recorded at d0f4fc92), so reviewDecision is CHANGES_REQUESTED. That thread is theirs to clear and this sign-off does not clear it.

Signed: Ankur-singh

❌❌❌ REJECTED ❌❌❌

@Ankur-singh — blocking: this single-node vLLM recipe has no merged upstream recipe documentation linked (checklist item 11, left unchecked in the sign-off, which confirms none exists). Everything else independently verifies green, including sweep + evals via run 30849838984 on in-PR commit 8767d90c.

✅ Check 0 (CODEOWNER): PASS — @ankur-singh is a listed owner of configs/nvidia-master.yaml; the other changed paths carry only the catch-all, which a recognized CODEOWNER satisfies.
✅ Check 1 (passing sweep on in-PR commit): PASS — commit 8767d90c (still in this PR) carries run 30849838984: all 16 agentic / lanes plus agentic eval / executed with conclusion success (not skipped), and the PR's config block + benchmark script are byte-identical between 8767d90c and this head. Informational: that run used the pre-agentx-v1.0.1 utils/aiperf pin; this head pins v1.0.1.
✅ Check 2 (evals pass): PASS — GSM8K em_strict 0.9689 (n=1319) ≥ the 0.90 bar in utils/evals/thresholds.yaml, produced on the same image as this PR's config (vllm/vllm-openai:nightly-5e35a6f4…).
❌ Check 3 (recipe linked + MERGED upstream): FAIL — no link to a merged PR in vllm-project/recipes or a published recipe page in the sign-off's additional detail section; the single-node recipe-documentation requirement applies (multinode: false, framework: vllm) and requires a MERGED upstream recipe before this PR merges.
✅ Check 4 (reuse command): PASS — /reuse-sweep-run 30849838984 posted by cquil11 (MEMBER), after the earlier bare-command revocation.
✅ Check 5 (latest checklist template): PASS — every current-template item is present; the unchecked items (3, 4, 11) are explained in the additional detail section.
✅ Check 6 (upstream image / engine-first ordering): PASS — vllm/vllm-openai:nightly-5e35a6f4… is from the upstream vLLM Docker org, and this entry is itself the vLLM submission for minimaxm3 on B300.
✅ Check 7 (MODELS.md): PASS — MiniMax-M3 agentic coding is active as "EAGLE3 only from 2026-08-03" and this PR is the EAGLE3 arm (Inferact/MiniMax-M3-EAGLE3-GQA); review date 2026-08-04.
✅ Check 8 (no architecture hacks): PASS — no --hf-overrides/model-override args; flags are attention-backend, KV, cudagraph, and scheduling knobs (--language-model-only matches the existing MiniMax-M3 recipes).
✅ Check 9 (spec-decode via chat templates): PASS — agentic trace replay is chat-formatted and the server sets --default-chat-template-kwargs '{"thinking_mode":"enabled"}'.
✅ Check 10 (no engine patches): PASS — no .patch/git apply, heredoc rewrites, site-packages edits, or forked engine wheels; the script only downloads models and installs harness deps.
✅ Check 11 (agentic golden AL): PASS — "rejection_sample_method":"synthetic" with "synthetic_acceptance_length":2.78, matching golden_al_distribution/minimaxm3_eagle3_gqa.yaml → minimax-m3.thinking_on[3] = 2.78 for the GQA draft head at 3 speculative tokens with thinking enabled.

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30938198455
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30938198455

vLLM recipe merged: vllm-project/recipes#730